Papers with convolutional neural networks
Scalable Wide and Deep Learning for Computer Assisted Coding (N18-3)
Copied to clipboard
Marilisa Amoia, Frank Diehl, Jesus Gimenez, Joel Pinto, Raphael Schumann, Fabian Stemmer, Paul Vozila, Yi Zhang
| Challenge: | In recent years the use of electronic medical records has accelerated resulting in large volumes of medical data when a patient visits a healthcare facility. |
| Approach: | They propose to use convolutional neural networks and logistic regression to build a machine learning based system for predicting ICD-10 codes from electronic medical records. |
| Outcome: | The proposed system can predict ICD-10 codes from electronic medical records using convolutional neural networks and logistic regression models. |
Delexicalized Paraphrase Generation (2020.coling-industry)
Copied to clipboard
| Challenge: | Using convolutional neural networks, we generate delexicalized sentences . 1.29% accuracy is achieved with the generated paraphrases . |
| Approach: | They propose a neural paraphrasing model that generates delexicalized sentences . they use convolutional neural networks to pool on slot values and use pointers to locate them . |
| Outcome: | The proposed model generates delexicalized sentences with high quality . it can be used for intent classification and named entity recognition tasks . |
Benchmarks and models for entity-oriented polarity detection (N18-3)
Copied to clipboard
| Challenge: | a dataset of 17,000 manually labeled documents is large for determining entity-oriented polarity in business news. |
| Approach: | They propose a convolutional neural network-based approach to classify entity-oriented polarity in business news. |
| Outcome: | The proposed model is based on convolutional neural networks and is small on the scale of existing models. |
YNU-junyi in BioNLP-OST 2019: Using CNN-LSTM Model with Embeddings for SeeDev Binary Event Extraction (D19-57)
Copied to clipboard
| Challenge: | BioNLP 2019 Shared Tasks: binary relation extraction of SeeDev task . Biological information extraction (Bio-IE) is a new field of research . |
| Approach: | They propose to use convolutional neural networks and long short term memory networks to construct a binary relation extraction model. |
| Outcome: | The proposed method performed well in the binary relation extraction task. |
Extending Event Detection to New Types with Learning from Keywords (D19-55)
Copied to clipboard
| Challenge: | Existing methods for event detection classify words or phrases into specific types of interest. |
| Approach: | They propose a new event detection formulation that describes types via keywords to match contexts in documents. |
| Outcome: | The proposed formulation improves the performance of the proposed model to new types. |
Transfer Learning for Context-Aware Question Matching in Information-seeking Conversations in E-commerce (P18-2)
Copied to clipboard
| Challenge: | Recent researches focus on deep learning and reinforcement learning for multi-turn information seeking conversation systems. |
| Approach: | They propose an efficient and effective multi-turn conversation model based on convolutional neural networks and extend it to adapt the knowledge learned from a resource-rich domain to enhance the performance. |
| Outcome: | The proposed model performs better than the existing model on an industrial chatbot called AliMe Assist. |
Self-Attention Enhanced CNNs and Collaborative Curriculum Learning for Distantly Supervised Relation Extraction (D19-1)
Copied to clipboard
| Challenge: | Distantly Supervised Relation Extraction (DSRE) suffers from mislabelled data . human annotation on large datasets is costly and often impossible . |
| Approach: | They propose a model that employs a collaborative curriculum learning framework to reduce mislabelled data. |
| Outcome: | The proposed model outperforms baselines including state-of-the-art in terms of P@N and PR curve metrics on a widely-used public dataset. |
On Orthogonality Constraints for Transformers (2021.acl-short)
Copied to clipboard
Aston Zhang, Alvin Chan, Yi Tay, Jie Fu, Shuohang Wang, Shuai Zhang, Huajie Shao, Shuochao Yao, Roy Ka-Wei Lee
| Challenge: | a dedicated study on orthogonality constraints for transformers has been lacking . plug-and-play constraints increase the BLEU of transformers . |
| Approach: | They propose to use plug-and-play constraints to encourage matrices to be orthogonal for numerical stability. |
| Outcome: | The proposed constraint increases the BLEU on the large-scale WMT’16 EnDe benchmark by a factor of 28.4 to 29.6. |
Answering Legal Questions by Learning Neural Attentive Text Representation (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for retrieval-based question answering are limited by legal documents and long and complicated documents. |
| Approach: | They propose a retrieval-based model for answering legal questions at the article level by learning neural attentive text representation. |
| Outcome: | The proposed model outperforms state-of-the-art retrieval-based methods on an annotated corpus of 5,922 Vietnamese legal questions in terms of recall and NDCG. |
Enhancing Drug-Drug Interaction Extraction from Texts by Molecular Structure Information (P18-2)
Copied to clipboard
| Challenge: | Graph Convolutional Networks (GCNs) can extract drug-drug interactions (DDIs) from texts using external drug molecular structure information. |
| Approach: | They propose a novel neural method to extract drug-drug interactions (DDIs) from texts using external drug molecular structure information. |
| Outcome: | The proposed model can extract drug-drug interactions (DDIs) from texts with high accuracy and the molecular information can enhance text-based extraction by 2.39 percent points in the F-score on the DDIExtraction 2013 shared task data set. |
Convolutional Neural Networks with Recurrent Neural Filters (D18-1)
Copied to clipboard
| Challenge: | Convolutional neural networks (CNNs) use recurrent neural networks as convolution filters to capture language compositionality and long-term dependencies. |
| Approach: | They propose to use recurrent neural networks (RNNs) as convolution filters to capture language compositionality and long-term dependencies. |
| Outcome: | The proposed convolutional neural networks achieve state-of-the-art on two sentences and the Stanford Sentiment Treebank. |
Parameterized Convolutional Neural Networks for Aspect Level Sentiment Classification (D18-1)
Copied to clipboard
| Challenge: | Existing approaches to aspect level sentiment classification ignore aspect information, causing large error. |
| Approach: | They propose a parameterized convolutional neural network for aspect level sentiment classification . they incorporate aspect information into convolutionally-based neural networks . |
| Outcome: | The proposed model achieves excellent results on SemEval 2014 datasets. |
Rational Recurrences (D18-1)
Copied to clipboard
| Challenge: | Recent studies show that neural models lack strong intuitions . recent studies show connections between convolutional neural networks and weighted finite state automata (WFSAs) |
| Approach: | They show that some recurrent neural networks share a connection to weighted finite state automata (WFSAs) they define rational recurrences as recursive hidden state update functions . they propose to use these functions to write forward calculations of a finite set of WFSA's . |
| Outcome: | The proposed model outperforms two baselines on language modeling and text classification. |
CASCADE: Contextual Sarcasm Detection in Online Discussion Forums (C18-1)
Copied to clipboard
| Challenge: | Existing studies on sarcasm detection focus on lexical, syntactic and semantic cues, but sarcasm can be expressed implicitly without such cue. |
| Approach: | They propose a ContextuAl SarCasm DEtector which extracts contextual information from the discourse of a discussion thread. |
| Outcome: | The proposed model improves on a large Reddit corpus. |
Do Neural Topic Models Really Need Dropout? Analysis of the Effect of Dropout in Topic Modeling (2023.eacl-main)
Copied to clipboard
| Challenge: | Dropout is a regularization trick used to resolve overfitting in large feedforward neural networks, but there is nil analysis of it for unsupervised models and in particular, VAE-based neural topic models. |
| Approach: | They propose to use dropout to solve overfitting problems in unsupervised neural topic models by stochastically dropping out the activation of neurons to prevent complex co-adaptations of feature vectors. |
| Outcome: | The proposed class of neural topic models can be used to improve the quality and predictive performance of the generated topics. |
A Probabilistic Model for Joint Learning of Word Embeddings from Texts and Images (D18-1)
Copied to clipboard
| Challenge: | Existing approaches combine language and perception to infer word embeddings . however, the embeddables produced by such models do not reflect the actual word representations. |
| Approach: | They propose a probabilistic model that integrates linguistic and perceptual inputs to explain observed word-context pairs in a text corpus. |
| Outcome: | The proposed model achieves competitive or stronger results on tasks of assessing pairwise word similarity and image/caption retrieval compared to other state-of-the-art models. |
VICTOR: a Dataset for Brazilian Legal Documents Classification (2020.lrec-1)
Copied to clipboard
Pedro Henrique Luz de Araujo, Teófilo Emídio de Campos, Fabricio Ataides Braz, Nilton Correia da Silva
| Challenge: | Approximately 10% of these are unstructured and requiring a lot of time to sort through. |
| Approach: | They propose to use a dataset built from Brazil's Supreme Court digitalized legal documents to improve document type classification and theme assignment tasks. |
| Outcome: | The proposed dataset is based on 45 thousand appeals and contains roughly 692 thousand documents—about 4.6 million pages. |
Challenging Reading Comprehension on Daily Conversation: Passage Completion on Multiparty Dialog (N18-1)
Copied to clipboard
| Challenge: | Existing approaches to reading comprehension on multiparty dialogs have focused on children's stories or newswire. |
| Approach: | They propose a new corpus and a robust deep learning architecture for a task in reading comprehension on multiparty dialog. |
| Outcome: | The proposed model outperforms the state-of-the-art model on a different genre using bidirectional LSTM, showing a 13.0+% improvement for longer dialogs. |
Rethinking Network Pruning – under the Pre-train and Fine-tune Paradigm (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing pruning results on benchmark transformers, such as BERT, are not as remarkable as those of convolutional neural networks. |
| Approach: | They propose to apply a knowledge-aware pruning process to transformer-based pre-trained language models to reduce model size and model weight. |
| Outcome: | The proposed pruning method outperforms the leading competitors with a 20-times weight/FLOPs compression and neglectable loss in prediction accuracy. |
EMR Coding with Semi-Parametric Multi-Head Matching Networks (N18-1)
Copied to clipboard
| Challenge: | Electronic medical record (EMR) coding is the process of extracting diagnosis and procedure codes from the digital record (the EMR) pertaining to a patient's visit. |
| Approach: | They propose a neural network architecture that combines ideas from few-shot learning matching networks, multi-label loss functions, and convolutional neural networks for text classification to significantly outperform other state-of-the-art models. |
| Outcome: | The proposed model outperforms existing models on a well known de-identified EMR dataset with multi-label performance measures. |
Don’t Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization (D18-1)
Copied to clipboard
| Challenge: | Existing approaches to summarize documents are not extractive and require an abstractive approach. |
| Approach: | They propose a novel abstractive model which is conditioned on the article’s topics and based entirely on convolutional neural networks. |
| Outcome: | The proposed model outperforms an oracle extractive system and state-of-the-art abstractive approaches when evaluated automatically and by humans. |
Aspect Based Sentiment Analysis with Gated Convolutional Networks (P18-1)
Copied to clipboard
| Challenge: | Aspect-based sentiment analysis can provide more detailed information than general sentiment analysis. |
| Approach: | They propose a model based on convolutional neural networks and gating mechanisms which can selectively output the sentiment features according to the given aspect or entity. |
| Outcome: | The proposed model can selectively output sentiment features according to the given aspect or entity. |
Non-Autoregressive Semantic Parsing for Compositional Task-Oriented Dialog (2021.naacl-main)
Copied to clipboard
| Challenge: | Semantic parsing using sequence-to-sequence models is stymied by higher compute requirements and higher latency. |
| Approach: | They propose a non-autoregressive approach to predict semantic parse trees with an efficient seq2seq model architecture. |
| Outcome: | The proposed architecture achieves an 81% reduction in latency on TOP dataset and retains competitive performance over non-pretrained models on three different semantic parsing datasets. |
Transformer-Exclusive Cross-Modal Representation for Vision and Language (2021.findings-acl)
Copied to clipboard
| Challenge: | a number of approaches to crossmodal representation have been used, but transformer architecture has taken over the recurrent neural networks in natural language processing tasks. |
| Approach: | They propose to use transformer architecture to handle cross-modal representations for vision and language with compatible performance to convolutional neural networks. |
| Outcome: | The proposed model outperforms recurrent neural networks in vision and language representations with transformer architecture. |
Enhancing Air Quality Prediction with Social Media and Natural Language Processing (P19-1)
Copied to clipboard
| Challenge: | predicting air quality is a major concern for human health, but the changes of air quality conditions are still difficult to monitor. |
| Approach: | They propose to exploit social media and natural language processing techniques to enhance air quality prediction. |
| Outcome: | The proposed approach improves air quality prediction over baseline that does not use social media by 6.9% to 17.7% in macro-F1 scores. |
Adaptive Convolution for Text Classification (N19-1)
Copied to clipboard
| Challenge: | Existing convolutional neural networks (CNNs) use sparse representations of text, such as bag-of-words. |
| Approach: | They propose an adaptive convolution for text classification to give flexibility to convolutional neural networks (CNNs) they attach filter-generating networks to convevolution blocks in existing CNNs . |
| Outcome: | The proposed convolution improves performance in seven benchmark datasets by 2.6 percentage points . the proposed conversions can be likened to players of the twenty questions . |
Explaining Character-Aware Neural Networks for Word-Level Prediction: Do They Discover Linguistic Rules? (D18-1)
Copied to clipboard
| Challenge: | Character-level features are used in many natural language processing algorithms but little is known about the character-level patterns they learn. |
| Approach: | They extend contextual decomposition technique to convolutional neural networks and bidirectional long-term memory networks to evaluate and compare these models for morphological tagging on three morphology-dependent languages. |
| Outcome: | The proposed models implicitly discover understandable linguistic rules for morphological tagging on three morphology-dependent languages. |
Navigating the Kaleidoscope of COVID-19 Misinformation Using Deep Learning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Various deep learning models have been successfully employed for this type of NLP task of text classification. |
| Approach: | They propose a mixed-domain transfer learning approach that only captures local context and exhibits poor generalization. |
| Outcome: | The proposed model captures local and global contexts, but lacks generalization . a combination of shallow network-based domain-specific models and convolutional neural networks can extract local and globally context directly from the target data in a hierarchical fashion, enabling it to offer a more generalizable solution. |
Page Stream Segmentation with Convolutional Neural Nets Combining Textual and Visual Features (L18-1)
Copied to clipboard
| Challenge: | (retro-)digitizing paper-based files is a major undertaking for private and public archives and an important task in electronic mailroom applications. |
| Approach: | They propose to use convolutional neural networks to combine image and text features to achieve optimal document separation. |
| Outcome: | The proposed approach achieves an accuracy of 93 % and is considered a state-of-the-art for this task. |
Learning with Noisy Labels for Sentence-level Sentiment Classification (D19-1)
Copied to clipboard
| Challenge: | Existing research on learning with noisy labels dates back to the 1980s, but it is still vibrant today. |
| Approach: | They propose a novel DNN model called NetAb to deal with noisy labels during training and train the networks using their respective loss functions in mutual reinforcement. |
| Outcome: | The proposed model can fit training data with noisy labels and predict clean labels. |
Ontology Matching Using Convolutional Neural Networks (2020.lrec-1)
Copied to clipboard
| Challenge: | a growing number of ontologies require effective ways to align different ontology models . traditional methods to align ontological models are based on string metrics and structure analysis . but convolutional neural networks can be applied as-is to any domain, allowing for cross-domain applications . |
| Approach: | They propose a method to align ontologies automatically using machine learning techniques . they use convolutional neural networks to perform string matching between class labels . |
| Outcome: | The proposed method achieves state-of-the-art on ontologies from the Ontology Alignment Evaluation Initiative (OAEI) it maintains good performance when tested on a different domain, which could lead to cross-domain applications. |
Discovering influential text using convolutional neural networks (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for estimating the effects of text on human evaluation are limited to testing a small number of pre-specified text treatments. |
| Approach: | They propose a method for flexibly discovering clusters of similar text phrases that are predictive of human reactions to texts using convolutional neural networks. |
| Outcome: | The proposed method can detect and predict human reactions to texts under certain assumptions. |
LETS-C: Leveraging Text Embedding for Time Series Classification (2025.acl-long)
Copied to clipboard
| Challenge: | Recent advances in language modeling have shown promising results when applied to time series data. |
| Approach: | They propose a method to fine-tune large language models for time series classification tasks using text embedding models and a simple classification head. |
| Outcome: | The proposed model outperforms the current SOTA model on a time series classification benchmark and uses only 14.5% of the trainable parameters. |